njtr&.n rarro*  ru*  r  _>r  r  « «^*. /-.•/». i  u„. 


NAMRL 


TOE  EFFECTS  OF  VOCAL  VERSUS  MANUAL  RESPONSE 
MODALITIES  CN  MULTI-TASK  PERFORMANCE 


G.  R.  Griffin  and  J.  D.  Mosko 


DT1C 

ELECTE 
OCT  7  1985 


February  1985 


-te  R  O'SPA  C-.E  M'E-otc  At*  RE  <1  1*  r  H  .1 


E  N  S  A  (f :  A  . '  f  l  (JP.£  A  . 


Approved  for  public  release;  distribution  unlimited 


Approved  for  public  release;  distribution  unlimited 


THE  EFFECTS  OF  VOCAL  VERSUS  MANUAL  RESPONSE 
MODALITIES  ON  MULTI-TASK  PERFORMANCE 


G.  R.  Griffin  and  J.  D.  Mosko* 


Naval  Medical  Research  and  Development  Command 
62758N  MF58528  MG5852801B  0001 

Naval  Air  Systems  Command 
61153N  WR0 4210. WR0 421001.6142 


Reviewed  by 

F.  E.  Guedry,  Ph.D 
Senior  Scientist 

DT1C 

SELECTE 
OCT  7  1985 

February  198  5  rj 


Approved  and  Released  by 

Captain  W.  M.  Houk,  MC  USN 
Commanding  Officer 


Naval  Aerospace  Medical  Research  Laboratory 
Naval  Air  Station,  Pensacola,  Florida  32508-5700 


I 


SUMMARY  PAGE 


THE  PROBLEM 

The  increasing  complexity  of  display  and  control 
instrumentation  in  modern  high  performance  aircraft  has  the 
potential  to  overload  the  human  operator  and  result  in  diminished 
system  performance.  Interactive  voice  technology  has  been 
proposed,  as  a  method  to  reduce  the  high  workload  placed  on  the 
pilots  of  military  aircraft.  This  report  presents  the  results  of 
an  experiment  designed  to  evaluate  the  effects  on  human 
performance  of  vocal  versus  manual  response  modalities  on  single 
and  multiple  tasks  simulating  some  conditions  of  flight. 

FINDINGS 

Results  indicated  a  significant  increase  in  performance 
precision  on  a  psychomotor  task  when  a  vocal,  rather  than  a 
manual  response  mode  was  used  in  the  simultaneous  performance  of 
multiple  tasks.  These  results  suggest  that  human  performance  on 
visually  oriented  multiple  tasks  requiring  simultaneous  execution 
may  be  improved  if  some  of  the  work  effort  can  be  performed  using 
a  vocal  input/output. 
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INTRODUCTION 

The  increasing  complexity  of  display  and  control 
instrumentation  in  modern,  high  performance  aircraft  holds 
considerable  potential  for  overloading  the  sensory/motor  systems 
of  the  human  operator  and  diminishing  system  performance. 
Interactive  voice  technology  has  been  proposed  to  alleviate  the 
information  flow  burden  by  redistributing  certain  information 
display  and  control  functions  from  visual-manual  to  auditory- 
vocal,  perceptual-motor  mechanisms.  Voice  technology  has  the 
potential  to  improve  human  performance  in  a  variety  of  tasks 
which  are  now  performed  with  the  eyes  and  hands.  In  high  demand 
aviation  environments,  for  example,  voice  technology  would  allow 
operators  to  interact  vocally  witn  an  onboard  computer,  in  order 
to  select  weapons,  switch  radar  modes  and  radio  frequencies, 
locate  and  paint  enemy  targets,  release  chaff  and  flares,  and 
acquire  information  auditorily  (e.g.,  threat  location  and 
warnings,  airspeed,  altitudes,  g-force,  closing  rates,  fuel 
state),  while  maintaining  a  visual  orientation  outside  the 
cockpit. 

The  successful  implementation  of  interactive  voice  devices 
in  work  environments  depends  on  equipment  reliability,  user 
acceptance,  and  most  importantly,  the  effective  integration  of 
man  and  machine.  Interactive  voice  systems  must  not  impose 
performance  demands  on  the  operator  that  would  interfere  with 
concurrent  activities.  Not  only  must  interactive  voice  devices 
be  effective  in  complex  noise  environments,  they  must  also  be 
insensitive  to  changes  in  the  operator's  voice  characteristics 
which  result  from  gravitational  forces,  vibration,  workload  and 
other  inflight  stressors.  The  present  effort  sought  to  compare 
the  efficiency  of  vocal  versus  manual  responses  to  one  task  while 
the  operator  simultaneously  executed  two  continuous,  compensatory 
tracking  tasks. 


METHOD 


SUBJECTS 

Sixty  student  naval  aviator  volunteers  participated  in  the 
study.  Fifty-six  subjects,  16  Marine  lieutenants  and  40  Navy 
ensigns,  were  awaiting  entry  into  the  primary  portion  of  the 
flight  training  program  at  Whiting  Field,  Florida.  Three 
subjects  were  Navy  ensigns  preparing  to  enter  the  Naval  Flight 
Officer  training  program.  One  additional  subject  had  recently 
completed  the  Navy  training  program  in  the  helicopter  flight 
training  pipeline. 

APPARATUS  AND  PROCEDURE 

Selected  as  tasks  for  the  experiment  were  a  complex 
psychorootor  task  (PMT)  (requiring  coordination  of  a  subject's 
right  hand,  eyes  and  feet  to  position  two  visual  cursors  on  a 
cathode  ray  tube)  and  a  dichotic  listening  task  (DLT)  (requiring 
selective  attention,  left  hand  finger  positioning,  and  vocal 
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responses  to  an  auditory  presentation).  Subjects  first 
performed  the  PMT  and  DLT  tasks  separately  and  then  in 
combination. 

Psychomotor  Task  (PMT) .  A  psychomotor  task  (PMT)  was  used 
to  represent  the  visual/psychomotor  component  of  the  simulated 
flight  task.  Subjects  were  required  to  maintain  two 
electronically  configured  cursors  on  a  fixed  target  of  a  CRT, 
using  the  floor-mounted  control  stick  and  foot  pedals  of  a 
Systems  Research  Laboratory  Psychomotor  Test  Device  (1).  The 
subject  controlled  an  upper  cursor  with  the  control  stick  using 
the  right  hand,  and  a  lower  cursor  by  operating  rudder  pedals 
with  the  feet.  Performance  data  were  obtained  during  four 
consecutive,  5  minute  test  sessions,  separated  by  rest  periods  of 
90  second  duration,,  The  PMT  performance  measure  was  a  machine¬ 
generated  cumulative  error  score  for  each  5  minute  session.  The 
error  score  was  derived  automatically  from  .01  inch  deviations 
from  an  ideal  or  "target"  position  on  three  (x,  y,  and  z)  axes  of 
the  visual  display,  and  added  together  to  provide  a  "total  error" 
score.  Tape  recorded  instructions  were  presented  via  headphones. 
A  diagram  of  the  experimental  apparatus  is  at  Figure  1. 


Figure  1.  Experimental  Apparatus. 


Dichotic  Listening  Task  (DLT) .  A  dichotic  listening  task 
(DLT)  patterned  after  that  developed  by  Gopher  (2)  and  Gopher  and 
Kahneman  (3)  and  subsequently  modified  at  this  laboratory  (4)  was 
used  to  represent  the  com  muni  cat  ions  and  attention  management 
component  of  the  simulated  flight  task*  The  DLT  stimuli 
consisted  of  letter-digit  strings  which  were  presented 
dichotically  to  subjects  during  three  sets  of  24  listening 
trials.  The  subjects  were  instructed  to  maintain  attention  to 
one  ear  while  ignoring  the  other  ear  and  to  report  the  digits 
presented  to  the  designated  ear  in  the  sequence  of  their 
occurrence.  An  illustrative  DLT  trial  is  depicted  in  Figure  2. 


PART  £ 

Left  Ear  R8NSMY2GB7FL6RL5 

"Right"  (Vocal  Channel  "attend"  Command) 

Right  Ear  YL3SR4FZ9XF0FN1L 


PART  II 

Left  Ear  B  F  4  3  7  9 

"Left"  (Vocal  Channel  "attend"  Command) 
Right  Ear  G  L  1  5  6  2 


Figure  2.  DLT  Trial  Example 


A  different  subject  response  modality  was  used  for  each  set  of 
listening  trials  (written,  keypad,  or  vocal  response).  The  three 
sets  of  listening  trials  were  separated  by  90  seconds  during 
which  time  the  test  administrator  read  the  instructions  for  the 
subsequent  set  of  listening  trials.  A  dual-channel  tape  recorder 
and  binaural  headphones  were  used  to  present  the  letter-digit 
strings  at  a  listening  level  of  72  dB/Leq  (re:  ?0  M  Pa).  A 
clear  plastic  simulated  keypad  was  mounted  on  the  PMT  device  so 
that  it  was  centered  in  the  field  of  view  horizontally  and 
vertically  (design  eye  level--position  "one")  and  at  a  lower  (30° 
below  design  eye),  but  horizontally  centered  location  (position 
"two").  Half  of  the  subjects  used  keypad  position  "one"  while 
the  remainder  used  keypad  position  "two"  throughout  all 
experimental  conditions.  For  separate  and  combined  task 
conditions,  the  subjects  were  instructed  to  make  keypad  responses 
with  the  left  hand,  while  maintaining  the  right  hand  on  the  PMT 
control  stick. 

Keypad  responses  were  visually  monitored  and  recorded  on 
magnetic  tape  by  the  test  administrator,  vocal  responses  by  the 
subjects  were  tape  recorded  for  later  analysis.  Written 
responses  were  marked  by  the  subjects  on  an  answer  sheet.  The 
performance  measure  for  the  DLT  was  the  average  number  of  correct 
responses  per  12  trials  for  each  response  method.,  DLT  response 
methods  and  order  of  presentation  of  the  PMT  and  DLT  were 
counterbalanced  across  subjects  and  experimental  conditions  in 


both  single-task  and  multi-task  performance.  A  subject  is  shown 
performing  in  the  Multi-task  condition  in  Figure  3. 


Figure  3.  A  subject  performing  in  the  multi-task  condition 


Multi-task  Performance  (PMT)  and  (DLT) .  Subjects  performed 
the  DLT  and  PMT  s  I  mu  1 t  an  e  o  u  s 1 y  (a  12  trial  DLT  and  PMT  session 
of  5  minute  duration).  The  onset  of  the  DLT  occurred  30  seconds 
after  the  onset  of  the  PMT  and  terminated  1.5  sec  before  PMT 
cessation.  The  subjects  used  keypad  and  vocal  responses 
alternately  in  performing  the  DLT. 


RESULTS 


SINGLE  TASK  PERFORMANCE 

Performance  means  and  standard  deviations  for  the  PMT  and 
DLT  single  tasks  are  presented  in  Table  I.  As  an  inspection  of 
Table  I  indicates,  there  was  substantial  improvement  in  PMT 
performance  over  repeated  sessions.  A  repeated  measures, 
analysis  of  variance  (see  summary  table  in  Appendix  A)  indicated 
session  1  (highest  error)  to  be  significantly  different,  (F  [3, 
177]  =  18.9,  p  <  .01)  from  sessions  2,  3,  and  4.  Scores  for  the 
latter  three  sessions  were  not  significantly  different  from  each 
other.  A  repeated  measures  analysis  of  variance  (see  summary 
table  in  Appendix  B)  also  revealed  significant  differences 
between  DLT  performance  means  (F  [2,  118]  =  12.9,  p  <  .01)  for 
the  three  response  modalities.  Written  responses  provided  the 
most  efficient  means  of  responding  to  the  DLT  task  followed  in 
order  by  keypad  and  vocal  response  modalities.  Written  and 
keypad  response  modalities  yielded  similar  performance  means,  and 
both  methods  resulted  in  significantly  higher  performance  than 
vocal  responses.  Practically,  however,  the  differences  were 
small.  A  paired  sample  t-test  (t  [58]  *  .24)  indicated  that 
keypad  location  on  the  PMT  device  did  not  significantly  affect 
single  task  DLT  performance. 


Table  I 

Single  Task  Performance 
Psychomotor  and  Dichotic  Listening 


Psychomotor  (PMT)  Dichotic  Listening  Task  (DLT) 

Total  Error  'Number  Correct  Responses* 


Session 

Mean 

S.D. 

Response  Method 

Mean 

S.  D. 

1 

18,254.1 

14070.2 

'written 

106.0 

2.2 

2 

11,804.2 

8078.2 

Keypad 

105.2 

2.7 

3 

10,458.4 

6151.7 

Vocal 

103.9 

4.6 

4 

10,318,1 

6759.0 

♦Average  number  correct  for  two  series  of  12  trials  (average 
number  correct  possible  *  108). 


MULTI-TASK  PERFORMANCE 


Following  single  task  performance  each  subject  performed  the 
DLT  and  PMT  simultaneously.  Mean  performance  scores  and  standard 
deviations  obtained  on  the  PMT  and  DLT  in  the  multi-task 
condition  are  depicted  in  Table  II.  Repeated  measures  analysis 
of  variance  (F  [2,  118]  =  45.1,  p  <  .01,  see  Appendix  C) 

indicated  a  significantly  lower  error  score  on  the  PMT  when  the 
vocal  response  modality  was  employed  for  the  DLT.  Finally  a 
repeated  measures  analysis  of  variance  (see  Appendix  D)  indicated 
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that  single  t  sk  DLT  keypad  performance  was  significantly 
improved  over  both  vocal  and  keypad  response  performance  in  the 
multitask  condition  (F  [3,  177]  *  11.44,  p  <  .01).  Keypad  and 
vocal  response  methods  in  the  multi-task  condition  and  single 
task  and  multitask  DLT  vocal  response  performance  was  not 
significantly  different. 

As  expected,  error  scores  for  later  sessions  of  the  single¬ 
task  PMT  were  lower  than  error  scores  for  the  multi-task  PMT. 
Similarly,  single-task  DLT  scores  obtained  with  the  keypad 
response  method  were  improved  over  keypad  DLT  scores  obtained  in 
the  multi-task  condition.  Unexpectedly,  DLT  vocal  response 
performance  was  not  significantly  different  in  the  single-task 
and  multi-task  conditions.  Finally,  paired  sample  t-tests 
indicated  that  keypad  location  had  no  significant  effect  on 
either  DLT  performance  (t  [58]  *  1.00)  or  PMT  performance  (t  [58] 
*  .52)  in  the  multi-task  condition. 


Table  II 

Multi-Task  Performance 
Psychomotor  and  Dichotic  Listening 


Dichotic  Listening  (DLT)  Psychomotor  (PMT) 


Number  Correct 

Responses 

Error  Scores 

Combined 

Response  Method 

Mean 

S.D. 

Mean 

S.D. 

Keypad 

101,6 

7.2 

25109.5 

19533.6 

Vocal 

102.4 

5.4 

15841.2 

15205.3 

DISCUSSION  AND  CONCLUSIONS 

The  data  support  an  expectation  of  improved  multi-task 
performance  if  selected  tasks  can  be  performed  through  vocal 
interaction.  In  this  experiment  subject  performance  deteriorated 
during  multi-task  performance  (compared  with  single-task 
performance)  regardless  of  DLT  response  modality.  However, 
deterioration  on  the  PMT  was  not  as  severe  when  a  vocal,  rather 
than  a  keypad,  response  modality  was  used  in  performing  the  DLT. 
Response  modality  did  not  affect  DLT  proficiency  in  the  multi¬ 
task  condition,  but  performance  using  keypad  responses  in  the 
multi-task  condition  was  poorer  than  keypad  response  performance 
performed  in  the  single-task  mode.  The  ability  of  the  subjects 
to  exhibit  DLT  performance  in  the  Multi-task  condition  (vocal 
response  modality)  comparable  to  single-task  DLT  performance  was 
an  unexpected  result.  Although  the  investigators  had  anticipated 
problems  with  masking  of  the  target  auditory  signals  by  the 
subjects'  own  vocal  responses,  none  of  the  subjects  reported  any 
serious  masking  difficulties  and  the  investigators  were  surprised 
by  the  ease  with  which  the  subjects  performed  the  DLT  (and  the 
PMT)  in  the  multi-task,  vocal  response  condition.  These  results 
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indicating  improved  multi-task  performance  using  voice  response, 
support  similar  findings  by  other  investigators  (5,  6,  7)  even 
though  the  experimental  tasks  employed  were  dissimilar. 

These  findings  indicate  a  significant  increase  in 
performance  precision  on  a  psychomotor  task  as  a  function  of  the 
use  of  a  vocal  response  method  for  a  second  simultaneously 
performed  task.  The  results  have  important  implications  for  the 
performance  of  multiple,  complex  aviation  tasks  which  are  highly 
visually  loaded,  and  lend  support  to  theory  (8,  9)  suggesting 
that  performance  on  multiple  visual  tasks  may  be  improved  if  some 
of  the  tasks  requiring  simultaneous  or  near  simultaneous 
execution,  can  be  performed  using  a  vocal  input/output  and  vice 
versa.  Examples  of  aviation  task  scenarios  which  could  possibly 
benefit  from  interactive  voice  technology  include  nap-of-the- 
earth  flight,  bombing  maneuvers,  anti-submarine  warfare  and  air 
combat  maneuvering. 
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APPENDIX  A 


Psychomotor  (PMT)  Single-Task  Performance  Summary 
(60  Subjects  -  Error  Scores  Combined) 


Session  Mean 

1  18254.13 

2  11804.18 

3  10458.42 

4  10318.12 


S.D. 

14070.18 

8078.18 

6151.67 

6758.99 


ANOVA 


Source 

DF 

SS 

MS 

F  Test 

Total 

239 

2299952.96 

Between 

'  59 

1253176.68 

Subjects 

Treatments 

3 

254090.78 

84696.93 

18.912 

Error 

177 

792685.49 

4478.44 

Sig.  Level 


.001** 


Summary  of  Simple  Contrasts 

Session  1  significantly  different  from  sessions  2,  3,  4. 
Sessions  2,  3,  and  4  were  not  significantly  different. 


APPENDIX  B 


Source  DF  SS  K.',  F  Test  Sig.  Level 

Total  179  2098.94 

Between  59  1333.11 

Subjects 

Treatments  2  137.26  68.63  12.88  .001** 

Error  118  628.57  5.33 


Summary  of  Simple  Contrasts 

Written  and  keypad  response  uethodo  significantly  different  from 
vocal  responses. 

Written  and  Keypad  response  methods  were  not  significantly 
different. 


APPENDIX  C 


Psychomotor  (PMT)  Single-Task  and  Multi-Task 
Performance  Comparison  (60  Subjects) 
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PMT  Single-Task  Performance  Mean  S.D. 

Session  4  10318.12  6758.99 


PMT  Multi-Task  Performance  Mean  S  u. 

With  Keypad  DLT  25109.47  1953^.56 

With  Vocal  DLT  15841.22  15205.26 


ANOVA 


Source 

DF 

SS 

MS 

F  Test  Siq. 

Level 

Total 

179 

4555177.24 

Between 

59 

3008002.39 

Subjects 

Treatments 

2 

670278.26 

335189.13 

45.110 

.001** 

Error 

118 

876796.59 

7430.48 

Summary  of  Simple  Contrasts 


All  performance  measures  are  significantly  different 


APPENDIX  D 


Dichotic  Listening  (DLT)  Single-Task  and  Multi-Task 
Performance  Summary  (60  Subjects) 


DLT  Response  Method 

Mean 

S.D. 

Keypad,  Single-Task 

105.25 

2.69 

Vocal,  Single-Task 

103.88 

4.61 

Keypad,  Multi-Task 

101.63 

7.21 

Vocal,  Multi-Task 

102.38 

5.37 

ANOVA 


Source 

DF 

SS 

MS 

F  Test  Sig.  Level 

Total 

239 

6903.08 

Between 

59 

4046.83 

Subjects 

Treatments 

3 

462.87 

154.61 

11.44  .001** 

Error 

177 

2392.43 

13.52 

Summary  of  Simple  Contrasts 

Single-Task  keypad  response  performance  significantly  different 
from  multi-task  performance  using  keypad  or  vocal  response 
methods. 

Single-task  vocal  response  performance  is  significantly  differen 
from  multi-task  keypad  reu^onse  performance. 

Single-task  vocal  response  and  multi-task  vocal  response 
performance  is  not  different. 

Multi-task  keypad  and  vocal  response  performance  is  not 
significantly  different. 
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